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DETAILED ACTION 

This is the initial response to the application filled June 19, 2003. Claims 1-66 are 
pending and are considered below. 

Claim Objections 

Claim 7 is objected to under 37 CFR 1.75 as being a substantial duplicate of 
claim 4. When two claims in an application are duplicates or else are so close in 
content that they both cover the same thing, despite a slight difference in wording, it is 
proper after allowing one claim to object to the other as being a substantial duplicate of 
the allowed claim. See MPEP § 706.03(k). Claim 7 recites the same limitation verbatim, 
as recited in claim 4. 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 1-8, 10-12, 14-17, 22, 23-31, 37-43, 45,46 and 48 are rejected under 35 

U.S.C. 103(a) as being unpatentable over Pitman (6,574,594) in view of Ellis 

(5,504,518). 
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As per claim 1 , Pitman discloses a computer-implemented process for providing an 
interactive user interface to a database of information describing contents of an input 
signal, comprising: extracting at least one fingerprint from each of at least one sample of 
at least one input signal (column 5 lines 10-16, for each piece of audio content received, 
a feature signature, or a key, is generated); comparing the extracted fingerprints to 
knov^^n fingerprints in a database of fingerprints of known signal objects to locate 
matching fingerprints for identifying one or more objects embedded in the at least one 
input signal as a known signal object (column 5 lines 16-18); providing an interactive 
user interface for querying the statistical information in the object database (column 3 
lines 1 8-20 and lines 50-60, a web browser is used to decode multimedia data 
streaming from the first and second servers). Pitman does not explicitly disclose in an 
object database, storing statistical information derived from the at least one input signal 
for each sample having an extracted fingerprint that matches a known fingerprint, and 
storing one or more extracted fingerprints that do not match any fingerprints in the 
database of fingerprints to the database of fingerprints as an unknown object fingerprint 
for use in matching subsequent instances of repeating objects in the at least one input 
signal. However, Pitman does disclose a user interface, and indicates that the system 
can be used to gather information about the identified audio segments (column 5 lines 
44-65, using identification and source information for each identified piece of 
information, a list is compiled which is used for market research purposes, popularity 
ratings, etc.). In addition, Ellis discloses storing one or more extracted fingerprints that 
do not match any fingerprints in the database (column 6 lines 16-22, a broadcast signal 
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is monitoiBd, and if it is determined to deviate from all known signals it is tagged as a 
new signal of interest, and a signature (fingerprint) is extracted and saved in a 
database). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to store statistical information derived from the at least one input signal 
in an object database, or supplemental temporary database, and store one or more 
extracted fingerprints that do not match any fingerprints in the database of fingerprints 
to the database of fingerprints as an unknown object fingerphnt in Pitman, since an 
object database, storing information pertaining to recently identified audio segments, 
would provide quick and efficient access to information about the recently identified 
audio segments; that information then used for market research purposes, as taught in 
Pitman (column 5 lines 60-61). In addition, saving fingerprints that don't match 
fingerprints in the database would enable the system to update its recognition database 
as indicated in Eliis (column 6 lines 7r8). 

As per claim 23, Pitman discloses a system for determining content of multiple media 
streams in real time, comprising: comparing each isignal fingerprint to a fingerprint 
database, said fingerprint database including known fingerprints of known media objects 
and metadata information describing the known media objects (column 5 lines 10-16, 
for each piece of audio content received, a feature signature, or a key, is generated, 
and column 5 lines 25-30, for each identified audio piece, audio content identification 
and source information are saved); identifying one or more media objects by locating 
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matching fingerprints of known media objects to each signal fingerprint (column 5 lines 
16-18); providing an interactive user interface for allowing at least one remote client 
computer to interact across a network with the object database residing on the at least 
one local server computer (column 3 lines 18-20 and lines 50-60, a web browser is used 
to decode multimedia data streaming from ttie first and second servers) . Pitman does 
not explicitly disclose populating an object database residing on at least one local server 
computer with statistical information derived from each identified media object, and with 
any metadata associated with the matching fingerprint of any known media objects, 
simultaneously monitoring two or more media streams in real time, sampling each 
media stream, and deriving a signal fingerprint from each sample from each media 
stream. However, Pitman does disclose a user interface, and indicates that the system 
can be used to gather information about the identified audio segments (column 5 lines 
44-65, using identification and source information (metatdata) for each identified piece 
of information, a list is compiled (statistical information) whicli is used for market 
research purposes, popularity ratings, etc.). Pitman also discloses deriving a signal 
fingerprint (signature or set of keys) for input media (column 5 lines 10-16, for each 
piece of audio content received, a feature signature, or a key, is generated). Ellis 
discloses simultaneously monitoring two or more media streams (column 8 lines 65-67 
and Figure 1 item 16, each local site monitors a broadcast signal for a particular 
geographical area, then communicates that information to a central site). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to simultaneously monitor two or more media streams in real time. 
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sample each media stream, and derive a signal fingerprint from each sample from each 
media stream in Pitman, since it would enable the system to gather broadcast - 
information from various geographical sites, as indicated in Ellis (column 8 lines 65-67). 

It would also have been obvious to one of ordinary skill in the art at the time of 
the invention to store statistical information derived from the at least one input signal in 
an object database in Pitman, since an object database, storing information pertaining 
to recently identified audio segments, would provide quick and efficient access to 
information about the recently identified audio segments; that information then used for 
market research purposes, as taught in Pitman (column 5 lines 60-61 ). 

As per claim 2, Pitman in view of Ellis disclose the computer-implemented process of 
claim 1 , and Pitman further discloses wherein the database of fingerprints includes 
metadata corresponding to information describing attributes of fingerprints in the 
database of fingerprints (column 5 lines 25-30, for each identified audio piece, audio 
content identification and source information (metadata) are saved). 

As per claim 3, Pitman in view of Ellis disclose the computer-implemented process of 
claim 2, however Pitman does not explicitly disclose further extracting any metadata 
and statistical information associated with any extracted fingerprints that match an entry 
in the database of fingerprints, and storing the extracted metadata to the object 
database for each sample having an extracted fingerprint that matches a fingerprint of a 
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known signal object. Pitman does indicate that identification information (metadata) is 
saved for every identified audio segment, and the system uses that information to 
determine other information of interest (column 5 lines 25-30 and lines 44-65, using 
identification and source information (metadata) for each identified piece of information, 
a list is compiled which is used for market research purposes, popularity ratings, etc.). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to extract and store metadata and statistical information in an object 
database in Pitman, since an object database, storing information pertaining to 
identified audio segments, would provide information for market research purposes, as 
taught in Pitman (column 5 lines 60-61) 

« 

As per claims 4,7, and 10, Pitman in view of Ellis disclose the computer-implemented 
process of claim 1, and P/7ma/i further suggests wherein the at least one input signal 
comprises one or more user selectable FM radio station broadcast signals in one or 
more geographic regions (column 3 lines 1-6, the system can accommodate any type of 
broadcast signal, including radio). 

As per claim 5, Pitman in view of Ellis disclose the computer-implemented process of 
claim 1, and Pitman further discloses wherein the at least one input signal comprises 
one or more television broadcast signals in one or more geographic regions (column 3 
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lines 1-6, the system can accommodate any type of broadcast signal, including 
television). 

As per claim 6, Pitman in view of Ellis disclose the computer-Implemented process of 
claim 1, and Pitman further discloses wherein the at least one input signal comprises 
one or more cable video signals in one or more geographic regions (column 3 lines 1-6, 
ttie system can accommodate any type of broadcast signal, including cable). 

As per claim 8, Pitman in view of Ellis disclose the computer-implemented process of | 
claim 1 , and Pitman further discloses wherein the at least one input signal comprises 
one or more Internet multimedia streams (column 3 lines 1-6, the system can 
accommodate any type of broadcast signal, including Internet). 

As per claim 1 1 , Pitman in view of Ellis disclose the computer-implemented process of 
claim 1 , and Pitman further discloses buffering the at least one input signal for a 
predetermined period of time (column 8 lines 35-37, events are produced from an audio 
signal, the event stored in a buffer until they are read by the key generator). 
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As per claim 12, Pitman in view of Ellis disclose the computer-implemented process of 
claim 1, and Pitman further discloses storing the at least one input signal on a computer 
readable medium (column 8 lines 31-37, events are produced from an audio signal, tlie 
event stored in memory sucli as RAM). 

As per claim 14, Pitman in view of Ellis disclose the computer-implemented process of 
claim 1 , and Pitman further disclose wherein the interactive user interface is a web- 
browser based user interface for performing user queries of the object database across 
the Internet (column 3 lines 48-55, a web browser such as Netscape Navigator). 

As per claim 15, Pitman in view of Ellis disclose the computer-implemented process of 
claim 1 , however Pitman does not explicitly disclose identifying one or more objects 
embedded in the at least one input signal as an unknown object by comparing the 
extracted fingerprints to unknown object fingerprints in the database of fingerprints of 
known and unknown signal objects to locate matching fingerprints: Ellis discloses 
identifying one or more objects embedded in the at least one input signal as an 
unknown object by comparing the extracted fingerprints to unknown object fingerprints 
in the database of fingerprints of known and unknown signal objects to locate matching 
fingerprints (column 6 lines 1 6-22, a broadcast signal is monitored, and if it is 
determined to deviate from all known signals it is tagged as a new signal of interest a^d 
a signature (fingerprint) is extracted and saved in a database). 
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Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to identify unknown object fingerprints in the database if Pitman, since it 
would enable the system to update its recognition database as indicated in Ellis 
(column 6 lines 7-8). 

As per claim 16 and 37, Pitman in view of Ellis disclose the computer-implemented 
process of claims 1 and 23, and Pitman further discloses wherein the interactive user 
interface for querying the statistical information in the object database comprises at 
least one predefined user selectable database query (column 5 lines 31-36, a copyright 
holder uses the system to automatically compile a list of broadcasts of its copyrighted 
content). 

As per claim 17, Pitman in view of Ellis disclose the computer-implemented process of 
claim 1, and Pitman further discloses wherein the object database is implemented on at 
least one local server computer and the interactive user interface is provided on at least 
one remote client computer accessible via the Internet for providing remote client 
interaction with the local object database (column 3 lines 7-20 and Figure 1 items 102 
and 112). 
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As per claim 18, Pitman in view of Ellis disclose the computer-implemented process of 
claim 1 , and Pitman further discloses automatically generating at least one set of 
information for characterizing at least one of the input signals, and automatically 
transmitting that set of information from a server computer to at least one client 
computer (column 11 lines 14-17, retrieval phase). 

As per claims 22 and 48, Pitman in view of Ellis disclose the system of claim 17, and 
Pitman further discloses a user interface for entering metadata associated each user 
specified signal sample (column 3 lines 18-20 and lines 50-60, a web browser is used to 
decode multimedia data streaming from the first and second servers). 

As per claim 24, Pitman in view of Ellis disclose the system of claim 23, and Pitman 
further discloses wherein the network is the Internet (column 2 line 66 - column 3 line 
2). 

As per claim 25, Pitman in view of Ellis disclose the system of claim 23, and Pitman 
further discloses wherein the media objects are any of songs, music, advertisements, 
commercials, station identifiers, speech audio clips, and emergency broadcast signals 
(Abstract, broadcast audio content). 
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As per claim 26, Pitman in view of Ellis disclose the system of claim 23, and.Pitman 
further suggests wherein the two or more media streams comprise television broadcast 
signals (column 3 lines 1-6, the system can accommodate any type of broadcast signal, 
including television). 

As per claim 27, Pitman in view of Ellis disclose the system of claim 23, and Pitman 
further discloses wherein the two or more media streams comprise cable multimedia 
broadcast signals (column 3 lines 1 -6, the system can accommodate any type of 
broadcast signal, including cable). 

As per claim 28, Pitman in view of Ellis disclose the system of claim 23, and Pitman 
further discloses wherein the two or more media streams comprise Internet multimedia 
streams (column 2 line 66 - column 3 line 2, the system monitors an audio broadcast 
over a network, such as the Internet). 

As per claim 29, Pitman in view of Ellis disclose the system of claim 23, and Pitman 
further suggests wherein the two or more media streams comprise automatically 
selected FM radio station broadcast signals in one or more geographic regions (column 
3 lines 1-6, the system can accommodate any type of broadcast signal, including radio 
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broadcast signals. Since FM is one type of radio signal, it must be able to nrionitor FM 
radio signals). 

As per claim 30, Pitman in view of Ellis disclose the system of claim 23, and Pitman 
further suggests wherein the two or more media streams comprise user selectable FM 
radio station broadcast signals in one or more geographic regions (column 3 lines 1-6, 
the system can accommodate any type of broadcast signal, including radio. Since FM is 
one type of radio signal, it must be able to monitor FM radio signals). 

As per claim 31 , Pitman in view of Ellis disclose the system of claim 23, and Ellis 
further discloses simultaneously applying separate fingerprint extraction engines to 
each media stream for deriving the signal fingerprint for each sample from each media 
stream (column 8 lines 65-67 and Figure 1 item 16, each local site monitors a broadcast 
signal for a particular geographical area, generates a fingerprint, then communicates 
that information to a central site). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to simultaneously apply separate fingerprint extraction engines to each 
media stream for deriving the signal fingerprint for each sample from each media 
stream in Pitman, since it would enable the system to simultaneously gather broadcast 
information from various geographical sites, as indicated in Ellis (column 8 lines 65-67). 
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As per claims 38,39,40 and 41 Pitman in view of Ellis disclose the system of claim 37, 
however Pitman does not explicitly disclose wherein the at least one predefined user 
selectable query includes a query for displaying user selectable music artist statistical 
information, statistical content information, and statistical commercial information with 
respect to one or more user selectable media streams. However, Pitman does disclose 
a system that can be used to automatically query broadcast content and gather 
statistical information (column 5 lines 30-36). In addition, broadcast content includes 
information on music artist information and commercials. 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to have a query that includes music artist statistical information in 
Pitman, since it would enable a copyright holder to automatically compile a list of 
broadcasts of its copyrighted material, and collect royalties based on the number of 
broadcasts of the copyrighted material, as taught in Pitman (column 5 lines 30-360. 

As per claim 42, Pitman in view of Ellis disclose the system of claim 23, and Pitman 

further discloses storing each monitored media stream for a predetermined period of 
time (column 8 lines 31 -37, the audio signal is stored in a buffer until it is read by the 
key generator process). 
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As per claim 43, Pitman in view of Ellis disclose the system claim 23, and Pitman 
further discloses wherein the interactive user interface further comprises a user 
selectable control for automatically providing a playback of at least one user selectable 
media object identified in one or more of the media streams (column 3 lines 56-61 , a 
web browser is used to decode the media stream and send it to the sound card and 
speakersl 

As per claim 45, Pitman in view of Ellis disclose the system of claim 23, however 
Pitman does not explicitly disclose automatically adding any fingerprints derived from 
each sample to the fingerprint database where the derived fingerprints do not match any 
signal fingerprints in the fingerprint database. Ellis discloses storing one or more 
extracted fingerprints that do not match any fingerprints in the database (column 6 lines 
16-22, a broadcast signal is monitored, and ifitjs determined to deviate from all known 
signals it is tagged as a new signal of interest, and a signature (fingerprint) is extracted 
and saved in a database) 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to automatically adding any fingerprints derived from each sample to the 
fingerprint database where the derived fingerprints do not match any signal fingerprints 
in the fingerprint database since it would enable the system to update its recognition 
database as indicated in Ellis (column 6 lines 7-8). 
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As per claim 46, Pitman in view of Ellis disclose the system of claim 23, however 
Pitman does not explicitly disclose automatically compiling a predetermined set of 
information from the object database, and automatically pushing that information from at 
least one of the local server computers to at least one of the remote client computers. 
However, Pitman does disclose that the system can be used to gather broadcast signal 
information from the database (column 5 lines 30-65). Performing this function 
automatically is simply automating a manual process. 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to automatically .compile a predetermined set of information from the 
object database, and automatically pushing that information from at least one of the 
local server computers to at least one of the remote client computers in Pitman, since a 
computer performing the automatic process, which would be used to determine 
licensing fees for the broadcast media, can perform function at a much greater speed 
then a human performing the manual process, this saving time and resources needed to 
gather the information. 

Claims 13,19,21 ,32,35.44,49 are rejected over Pitman in view of Ellis, further in 
view of Eubanks (6,704,553). 

As per claim 13, Pitman in view of Ellis disclose the computer-implemented process of 
claim 1, and P/ftnan further discloses extracting a sample fingerprint from each signal 



Application/Control Number: 10/600,589 Page 17 

Art Unit: 2626 

sample, adding each sample fingerprint to the database of fingerprints (column 5 lines 
10-16 and column, for each piece of audio content received, a feature signature, or a 
key. Incoming data is compared to keys in a database, so there must have been keys 
added to the database), and identifying one or more objects embedded in the at least 
one input signal by comparing the extracted fingerprints to sample fingerprints added to 
the database of fingerprints (column 5 lines 16-18). However neither Pitman nor Ellis 
disclose obtaining at least one signal sample via the user interface. Eubanl^s discloses 
a system for automatic control a radio turner, where the user can input preference 
information (column 5 lines 22-40, the user indicates their like or dislike of a current 
song being played). A system that enables user input is well known, as indicated by 
Eubanlis. 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time of the invention to obtain at least one signal sample via the user interface in 
Pitman and Ellis, since it would enable the user to design the fingerprint database and 
system controls according to their personal preferences, as indicated in Eubanlis, 
(column 1 lines 48-67). 

As per claim 19, Pitman in view of Ellis disclose the computer-implemented process of 
claim 1, however neither disclose wherein sampling of a particular input signal is 
suspended for a period of time when an object embedded in that input signal is 
identified in that signal, and where the period of time is either predetermined or is 
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determined by the characteristics of the identified object. Eubanks discloses a system 
that suspends sampling and playback of a signal for a period of time when an object 
embedded in the signal is identified (column 4 lines 5-20, the system determines if the 
template created by the signal matches a stored set of templates. If it does, the system 
sends a control signal to the tuner, which tunes to a new frequency). 

Therefore It would have been obvious to one of ordinary skill in the art at the time 
of the invention to suspend sampling of an input signal for a period of time when an 
object embedded in the signal is identified in Pitman and Ellis, since it would enable 
the system to skip songs, commercials, or other content that the user indicated a dislike 
of, as indicated in Eubanks (column 4 lines 17-19). 

As per claim 21 Pitman in view of Ellis disclose the computer-implemented process of 
claim 1 , however neither discloses a user interface for inputting at least one user 
specified signal sample, and wherein fingerprints are automatically extracted from each 
user specified signal sample and added to the known fingerprints in the database of 
fingerprints. Eubanks discloses a system for automatic control a radio turner, where the 
user can input preference information (column 5 lines 22-40, the user indicates their like 
or dislike of a current song being played). A system that enables user input is well 
known, as indicated by Eubanks. 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time of the invention to extract fingerprints from each user specified speech signal in 
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Pitman and Ellis, since it would enable the user to design the fingerprint database and 
system controls according to their personal preferences, as indicated in Eubanks, 
(column 1 lines 48-67). 

• 

As per claim 32 and 35, Pitman in view of Ellis disclose the system of claim 29, 
however neither discloses monitoring at least one automatically tunable receiver, 
wherein each automatically tunable receiver automatically switches between at least 
two radio broadcast streams at predefined intervals, and intervals that are defined by 
what has been identified, and samples each of the at least two radio broadcast streams 
for a predefined period of time. Eubanks discloses a system that switched between 
radio broadcasts at predetermined time intervals as well as intervals that are defined by 
what has been identified (column 5 lines 21-31 , the system outputs audio content until it 
receives a signal from the user indicating a like or a dislike and column 4 lines 5-20, the 
system determines if the template created by the signal matches a stored set of 
templates. If it does, the system sends a control signal to the tuner, which tunes to a 
new frequency). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to automatically switch between two radio broadcast streams based on 
predetermined time intervals, and time intervals that are defined by what has been 
identified in Pitman and Ellis, since it would enable the system to skip songs, 
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commercials, or other content that the user indicated a dislike of, as indicated in 
Eubanks (column 4 lines 17-19). 

As per claim 44, Pitman in view of Ellis disclose the system claim 23, however neither 
explicitly discloses wherein the interactive user interface further comprises a control for 
adding a user selectable media sample corresponding to a user identified media object, 
and wherein a signal fingerprint is automatically derived from the user selectable media 
sample and added to the fingerprint database along with user entered metadata for 
describing the user selectable media sample. Eubanks discloses a system for 
automatic control a radio turner, where the user can input preference information 
(column 5 lines 22-40, the user indicates their lif<e or dislike of a current song being 
played), A system that enables user input is well known, as indicated by Eubanks. 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time of the invention to extract fingerprints from each user specified media sample in 
Pitman and Ellis, since it would enable the user to design the fingerprint database and 
system controls according to their personal preferences, as indicated in Eubanks, 
(column 1 lines 48-67). 

As per claim 49, Pitman in view of Ellis disclose the system of claim 48, Pitman further 
discloses storing that fingerprint and any associated metadata in the fingerprint 
database (column 5 lines 10-16, for each piece of audio content received, a feature 
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signature, or a key, is generated, and column 5 lines 25-30, for each identified audio 
piece, audio content identification and source information are saved). However, neither 
Pitman nor Ellis explicitly discloses automatically deriving a signal fingerprint from each 
user media sample. Eubanks discloses a system for automatic control a radio turner, 
where the user can input preference information (column 5 lines 22-40, ttie user 
indicates their like or dislike of a current song being played). A system that enables 
user input is well known, as indicated by Eubanks, 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time of the invention to extract fingerprints from each user specified media sample in 
Pitman and Ellis, since it would enable the user to design the fingerprint database and 
system controls according to their personal preferences, as indicated in Eubanks, 
(column 1 lines 48-67). 

Claims 9,33,34 are rejected over Pitman in view of Ellis, further in view of 
0/eri/d (2,776,429). 

As per claim 9, Pitman in view of Ellis disclose the computer-implemented process of 
claim 4, however neither explicitly discloses multiplexing two or more FM radio station 
broadcast signals into a single input signal prior to extracting the at least one fingerprint 
from that single input signal. Olerud discloses a multiplex system for FM broadcast 
transmitters and receives (column 1 lines 15-19). Multiplexing two or more broadcast 
signals is well known in the art, as indicated by Olerud (column 1 lines 15-19). 
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Therefore it would have been obvious to one of ordinary sl<ill in the art at the time 
of the invention to multiplex two or more FM radio station broadcast signals in Pitman 
and Ellis, since it would enable the simultaneous transmission of one or more signals, 
as indicated in 0/erud (column 1 lines 19-22). 

As per claim 33, Pitman in view of Ellis disclose the system of claim 31, however 
neither explicitly discloses multiplexing the samples from each automatically tunable 
receiver into a separate multiplexed radio broadcast stream. O/eri/cf discloses a 
multiplex system for FM broadcast transmitters and receives (column 1 lines 15-19). 
Multiplexing two or more broadcast signals is well known in the art, as indicated by 
0/eriid (column 1 lines 15-19). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to multiplex two or more FM radio station broadcast signals in Pitman 
and Ellis, since it would enable the simultaneous transmission of one or more signals, 
as indicated in O/eri/cf (column 1 lines 19-22). 

As per claim 34, Pitman in view of Ellis further in view of Olerud disclose the system of 
claim 33, and Ellis further discloses deriving the signal fingerprint for each sample from 
each radio stream comprises simultaneously operating a separate fingerprint extraction 
engine for each multiplexed radio broadcast stream for deriving the signal fingerprint for 
each sample comprising each multiplexed radio broadcast stream column 8 lines 65-67 
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and Figure 1 item 16, each local site monitors a broadcast signal for a particular 
geographical area, generates a fingerprint then communicates that information to a 
central site). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to simultaneously applying separate fingerprint extraction engines to 
each multiplexed broadcast stream for deriving the signal fingerprint for each sample 
from each multiplexed broadcast stream in Pitman, since it would enable the system to 
simultaneously gather broadcast information from various geographical sites, as 
indicated in Ellis (column 8 lines 65-67). 

Cjaims 20 and 47 are rejected over Pitman in view of Ellis, further in view of 
Burges ("Extracting, Noise-Robust Features from Audio Data" IEEE ICASSP 2002). 

As per claim 20, Pitman in view of Ellis disclose the computer-implemented process of 
claim 1 , however neither disclose wherein extracting at least one fingerprint from each 
of at least one sample comprises performing a distortion discriminant analysis of each 
sample to generate fingerprints. Burges discloses a system that uses distortion 
discriminant analysis for audio fingerprinting (Abstract). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to extract at least one fingerprint from each of at least one sample 
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comprises performing a distortion discriminant analysis of each sample to generate 
fingerprints in Pitman and Ellis, since it provides an efficient feature extraction 
operation that produces robust features from the audio data, as indicated in Burges 
(Abstract). 

As per claim 47, Pitman in view of Ellis disclose the system of claim 23, however 
neither explicitly discloses wherein deriving a signal fingerprint from each sample from 
each media stream comprises applying a distortion discriminant analysis to each 
sample for deriving a trace representing a signal fingerprint from each sample. Burges 
discloses a system that uses distortion discriminant analysis for audio fingerprinting 
(Abstract). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to extract at least one fingerprint from each of at least one sample 
comprises performing a distortion discriminant analysis of each sample to generate 
fingerprints in Pitman and Ellis, since it provides an efficient feature extraction 
operation that produces robust features from the audio data, as indicated in Burges 
(Abstract). 

Claim 36 is rejected over Pitman in view of Ellis, further in view of Eubanks and 
further in view of Olerund, 
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As per claim 36, Pitman in view of Ellis further in view of Eubanks disclose the system 
of claim 35, however neither Pitman, Ellis nor Eubanks disclose multiplexing the 
samples from each automatically tunable receiver into a separate multiplexed radio 
broadcast stream. 0/erud discloses a multiplex system for FM broadcast transmitters 
and receives (column 1 lines 15-19). Multiplexing two or more broadcast signals is well 
known in the art, as indicated by Olerud (column 1 lines 15-19). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to multiplex the samples from each automatically tunable receiver into a 
separate multiplexed radio broadcast stream in Pitman and Ellis, since it would enable 
the simultaneous transmission of one or more signals, as indicated in O/erucf (column 1 
lines 19-22). 

Claims 50-56, 58-61 , 63-65 are rejected over Pitman in view of Ellis, further in 
view of Eubanks, 

As per claim 50, Pitman discloses a method for providing an interactive user interface 
for querying a database of content information that characterizes at least one signal, 
comprising: identifying one or more media objects in one or more of the broadcast 
signals by comparing each media object trace fingerprint to the known fingerprints in the 
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fingerprint database to locate matching fingerprints of known media objects (column 5 
lines 16-18); and providing an interactive user interface for allowing at least one remote 
client computer to interact across a network with the object database residing on the at 
least one local server computer (column 3 lines 18-20 and lines 50-60, a web browser is 
used to decode multimedia data streaming from the first and second servers). However, 
Pitman does not explicitly disclose monitoring at least one user selectable media 
broadcast signal common to a user selectable geographic region using at least one 
automatically tunable receiver, sampling each broadcast signal for any of predefined 
periods of time and for periods of time determined by media objects identified in the at 
least one broadcast signal; deriving a media object trace fingerprint from each sample, 
using a separate instance of a fingerprint engine for each receiver for simultaneously 
processing each monitored broadcast signal in real-time; comparing each media object 
trace fingerprint to a fingerprint database, said fingerprint database including known 
fingerprints of known media objects, metadata information describing the known media 
objects, and fingerprints of unknown objects for identifying unknown but repeated 
objects; populating an object database residing on at least one local server computer 
with statistical information derived from each identified media object, and with any 
metadata associated with the matching fingerprint of any known media objects; Ellis 
discloses deriving a media object trace fingerprint from each sample, using a separate 
instance of a fingerprint engine for each receiver for simultaneously processing each 
monitored broadcast signal in real-time (column 8 lines 65-67 and Figure 1 item 16, 
each local site monitors a broadcast signal for a particular geographical area, generates 
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a fingerprint, then communicates that information to a central s/te); comparing each 
media object trace fingerprint to a fingerprint database, said fingerprint database 
including known fingerprints of known media objects, metadata information describing 
the known media objects, and fingerprints of unknown objects for identifying unknown 
but repeated objects (column 6 lines 16-22, a broadcast signal is monitored, and if it is 
determined to deviate from all known signals it is tagged as a new signal of interest and 
a signature (fingerprint) is extracted and saved in a database). Eubanks discloses 
monitoring at least one user selectable media broadcast signal common to a user 
selectable geographic region using at least one automatically tunable receiver, sampling 
each broadcast signal for any of predefined periods of time and for periods of time 
determined by media objects identified in the at least one broadcast signal (column 5 
lines 21-31 , f/7e system outputs audio content until it receives a signal from the user 
indicating a like or a dislike and column 4 lines 5-20, the system determines if the 
template created by the signal matches a stored set of templates. If it does, the system 
sends a control signal to the tuner, which tunes to a new frequency). In addition, 
However, Pitman discloses a user interface, and indicates that the system can be used 
to gather information about the identified audio segments (column 5 lines 44-65, using 
identification and source information for each identified piece of information, a list is 
compiled which is used for market research purposes, popularity ratings, etc. 

Therefore it would have been obvious to one of ordinary skill in the art at the fime 
of the invenfion to simultaneously applying separate fingerprint extraction engines to 
each media stream for deriving the signal fingerprint for each sample from each media 
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stream, and identify unknown object fingerprints in Pitman, since it would enable the 
system to simultaneously gather broadcast information from various geographical sites, 
and update its recognition database as indicated in Ellis (column 8 lines 65-67 and 
column 6 lines 7-8). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to media streams based on predetermined time intervals, and time 
intervals that are defined by what has been identified in Pitman; since it would enable 
the system to skip songs, commercials, or other content that the user indicated a dislike 
of, as indicated in Eubanks (column 4 lines 17-19). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to store statistical information derived from the at least one input signal, 
and store one or more extracted fingerprints that do not match any fingerprints in the 
database of fingerprints to the database of fingerprints as an unknown object fingerprint 
in Pitman, since an object database, storing information about pertaining to identified 
audio segments, would provide information for market research purposes, as taught in 
Pitman (column 5 lines 60-61 ) 

As per claim 51 , Pitman in view of Ellis and further in view of Eubanks discloses the 
method of claim 50, however Pitman does not disclose wherein the unknown but 
repeated objects are themselves stored in an unknown object database, and further 
comprising analyzing the unknown but repeated objects to determine metadata 
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describing the unknown but repeating objects, said metadata then being entered in the 
fingerprint database. However, Pitman does disclose metadata stored in a fingerprint 
database corresponding to extracted fingerprints (column 5 lines 10-16, for each piece 
of audio content received, a feature signature, or a key, is generated, and column 5 
lines 25-30, for each identified audio piece, audio content identification and source 
information are saved). Ellis discloses storing unknown but repreated objects in the 
database (column 6 lines 16-22, a broadcast signal is monitored, and if it is determined 
to deviate from all known signals it is tagged as a new signal of interest, and a signature 
(fingerprint) is extracted and saved in a database). 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time of the invention to store unknown object fingerprints and their metadata in 
Pitman, since it would enable the system to simultaneously gather broadcast 
information from various geographical sites, and update its recognition database as 
indicated in Ellis (column 8 lines 65-67 and column 6 lines 7-8). 

As per claim 52, Pitman in view of Ellis and further in view of Eubanks disclose the 
method of claim 51, however neither Pitman, Ellis nor Eubanks explicitly disclose 
wherein analyzing the unknown but repeated objects to determine metadata comprises 
manual user identification and entry of the metadata for one or more of the repeated 
objects via the interactive user interface. Eubanks discloses a system for automatic 
control a radio turner, where the user can input preference information (column 5 lines 
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22-40, the user indicates their like or clislil<e of a current song being played). A system 
that enables user input is well known, as indicated by Eubanks. 

Therefore it would have been obvious to one of ordinary skill in the art at 
the time of the invention to enable manual user identification and entry of the metadata 
for one or more of the repeated objects via the interactive user interface in Pitman, 
since it would enable the user to design the fingerprint database and system controls 
according to their personal preferences, as indicated in Eubanks, (column 1 lines 48- 
67). 

As per claim 53, Pitman in view of Ellis and further in view of Eubanks disclose the 
method of claim 51 , however neither Pitman, Ellis or Eubanks explicitly discloses 
analyzing the unknown but repeated objects to determine metadata comprises 
automatically identifying the unknown but repeated objects by comparing the objects to 
at least one additional object database, and importing any metadata associated with the 
automatically identified unknown but repeated objects into the fingerprint database. 
However, Pitman does disclose a user interface, and indicates that the system can be 
used to gather information about the identified audio segments (column 5 lines 44-65, 
using identification and source infornr^ation for each identified piece of information, a list 
is compiled which is used for market research purposes, popularity ratings, etc.) using 
metadata extracted from fingerprints and stored in a database. In addition, Ellis 
discloses storing one or more extracted fingerprints that do not match any fingerprints in 
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the database (column 6 lines 16-22, a broadcast signal is monitored, and if it is 
determined to deviate from all known signals it is tagged as a new signal of interest and 
a signature (fingerprint) is extracted and saved in a database). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to analyze the unknown but repeated objects to determine metadata 
comprises automatically identifying the unknown but repeated objects by comparing the 
objects to at least one additional object database, and importing any metadata 
associated with the automatically identified unknown but repeated objects into the 
fingerprint database in Pitman, Ellis and Eubanks, since saving fingerprints that don't 
match fingerprints in the database would enable the system to update its recognition 
database as indicated in Ellis (column 6 lines 7-8). In addition, an object database, 
storing information about pertaining to identified audio segments, would provide quick 
access to information about recently recognized signals. 

As per claim 54, Pitman in view of Ellis and further in view of Eubanks disclose the 
method of claim 50, and Pitman further discloses wherein the at least one user 
selectable broadcast signal is a radio station (column 3 lines 1-6, the system can 
accommodate any type of broadcast signal, including radio). 

As per claim 55, Pitman in view of Ellis and further in view of Eubanks disclose the 
method of claim 50, and Pitman further discloses wherein the at least one user 
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selectable broadcast signal is any of a television broadcast signal, an internet broadcast 
signal, a network broadcast signal, and a cable broadcast signal (column 3 lines 1-6, 
the system can accommodate any type of broadcast signal, including television, internet 
and cable). 

As per. claim 56, Pitman in view of Ellis and further in view of Eubanks disclose the 
method of claim 50, and Pitman further discloses wherein the network is the Internet 
(column 2 line 66- column 3 line 5 and Figure 1). 

As per claim 58, Pitman in view of Ellis and further in view of Eubanks disclose the 
method of claim 50, Pitman further discloses wherein the interactive user interface 
comprises at least one predefined user selectable query for retrieving statistical 
information and metadata from the object database by user selection of the predefined 
user selectable query (column 5 lines 31-36, a copyright holder uses the system to 
automatically compile a list of broadcasts of its copyrighted content). 

As per claim 59, Pitman in view of Ellis and further In view of Eubanks disclose the 
method of claim 50, and Pitman further discloses wherein media objects include any of 
music, advertisements, commercials, station identifiers, speech audio clips, videos, and 
emergency broadcast signals (Abstract, broadcast audio content). 
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As per claim 60, Pitman in view of Ellis and further in view of Eubanks disclose the 
method of claim 50, and Pitman discloses wherein metadata includes any of radio 
station call signs, music titles, song titles, music artist names, music album titles, 
commercial titles, commercial product information, and video titles (column 5 lines 25- 
30, for each identified audio piece, audio content identification and broadcast source 
(station call sign) information are saved). 

As per claim 61 , Pitman in view of Ellis and further in view of Eubanks disclose the 
method of claim 50, and Pitman further discloses wherein statistical information 
includes any of media object play times, media object play dates, media object play 
Station, and media object number of plays (column 5 lines 41-52, a list is compiled of 
what audio content was broadcast by a particular broadcast station). 

As per claim 63, Pitman in view of Ellis and further in view of Eubanks disclose the 
method of claim 50, and Pitman further discloses wherein the interactive user interface 
is a web-browser based user interface (column 3 lines 18-20 and lines 50-60, a web 
browser is used to decode multimedia data streaming from the first and second 
servers). 
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As per claim 64, Pitman in view of Ellis and further in view of Eubanks disclose the 
method of claim 50, and Pitman further discloses automatically generating at least one 
report comprising statistical and metadata for describing one or more of the media 
broadcast signals (column 5 lines 31-36, a copyright holder uses the system to 
automatically compile a list of broadcasts of its copyrighted content). 

As per claim 65, Pitman in view of Ellis and further in view of Eubanks disclose the 
method of claim 64, further comprising automatically transmitting the at least one report 
to at least one of the remote client computers (column 5 lines 31-36 and Figure 1, a 
copyright holder uses the system to automatically compile a list of broadcasts of its 
copyrighted content, the copyright holder accessing the system through a client 
computer). 

Claim 66 is rejected ower Pitman in view of Ellis further in view of Eubanks and 
further in view of Surges, 

As per claim 66, Pitman in view of Ellis and further in view of Eubanks disclose the 
method of claim 50, however neither Pitman, Ellis nor Eubanks discloses wherein 
deriving the media object trace fingerprint from each sample comprises applying a 
distortion discriminant analysis to each sample for deriving a trace representing the 
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media object trace fingerprint from each sample. Burges discloses a system that uses 
distortion discriminant analysis for audio fingerprinting (Abstract). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to extract at least one fingerprint from each of at least one sample 
comprises performing a distortion discriminant analysis of each sample to generate 
fingerprints in Pitman, Ellis, and Eubanks since it provides an efficient feature 
extraction operation that produces robust features from the audio data, as indicated in 
Burges (Abstract). 

Claims 57 and 62 are rejected over Pitman in view of Ellis further in view of Eubanks 
and further in view oi IEEE ("IEEE 100 The Authoritative Dictionary of IEEE Standards 
Terms" IEEE 2000). 

Pitman in view of Ellis and further in view of Eubanks disclose the method of claims 50 
and 55, however neither Pitman, Ellis nor Eubanks disclose wherein the object 
database is an SQL database, and a user selectable query includes SQL queries for 
extracting information relating to the media objects, metadata, and statistical information 
in the object database. However, SQL is a standard language that provides an 
interface to relational database systems, as indicated by IEEE, 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to have an SQL database and provide SQL queries in Pitman, Ellis, 
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and Eubanks, since SQL is a reliable standard for database queries, thus removing the 
need to devote time and resources to developing a new method for database query. 



Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Dorothy Sarah Siedler whose telephone number is 571- 
270-1067. The examiner can normally be reached on Mon-Thur 9:30am-5:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richemond Dorvil can be reached on 571-272-7602. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



DSS 




PRIMARY EXAMINER 



